Exposure control method and system for an image capture device

ABSTRACT

Improved exposure control processes, devices, and systems are provided for wide angle lens imaging systems that suffer from image distortion. The exposure control uses a model of the wide angle lens distortion that estimates the weights that respective pixels of the image would have when producing an undistorted version of the image signal, and then scales pixel intensity values by the respective weights to produce weighted pixel values. The weighted pixel values are provided to an exposure control pixel counting process to produce one or more exposure feedback control signals, which control exposure features of the imaging system.

TECHNICAL FIELD OF THE INVENTION

The invention relates generally to the field of image capture and morespecifically to exposure control in an imaging system that captures adistorted image with an image capture device having a wide-angle lensthat distorts (warps) the field of view, and undistorts the distortedimage with image processing.

BACKGROUND OF THE INVENTION

The solid-state look-around (SSLA) endoscope incorporates verywide-angle optics which capture a large field of view. A SSLA endoscopetypically has an actual angle of view which is non-zero (meaning thescope lens is tilted relative to the axis of the scope rod). During aprocedure, a doctor will select a different display angle of view from arange of programmed possibilities. An unwarping and rotation algorithmmay then simulate an image as if the actual viewing angle were the sameas the selected display angle of view. In such simulation, a largeportion of the captured image (also called the sensor image) is notused, but rather only the pixels inside a region of interest (ROI) areused to form the displayed image. The captured image typically includeswarping or distortion due to the use of a wide angle lens. Suchdistortion may include highly non-linear barrel distortion in which theimage magnification increases non-linearly with the distance from theaxis of the optical system. The distortion typically needs to becorrected to view useful images from the endoscope. The correctionrequires processing the image using knowledge of the distortionresulting from the wide angle lens and other optical elements, typicallywith a relatively slow image processing technique that non-linearlyresamples the image to produce the unwarped displayed image data.

To obtain quality images using endoscopes, exoscopes, borescopes, orother imaging devices, the image exposure must be controlled. Thisinvolves adjusting elements such as the light source, light sourceintensity, the optical assembly aperture, and the shutter speed or timefor which image sensor integrates charge for each image frame. Mostprior art methods for exposure control include steps wherein the numberof image pixels that have a digital value above or below predeterminedthresholds are counted. If the number is too high, the exposure iscontrolled downward (faster exposure) to avoid overexposure, and if thenumber is too low, the exposure in controlled upward to avoidunderexposure. When device produces distorted images, it is theundistorted version that should best be used to produce accurate countsof the number of pixels above or below the thresholds, because theundistorted image is the image that is viewed, analyzed, and stored.However, prior-art imaging systems typically count pixels of thedistorted image, and therefore do not provide accurate pixel counts ofthe corresponding undistorted image. It is possible to count the pixelsin the undistorted image, but this causes delays in the result and canthus delay the automatic exposure control loop. U.S. Publication No.20140092226 by Kuriyama describes an automatic exposure control devicethat adjusts exposure values linearly based on a pixels distance fromthe center of an automatic exposure area. U.S. Publication No.20140081083 by Morita et al. describes an exposure control process thatcan calculate exposure to control brightness corresponding to aparticular area of a scope's image.

What is needed are improved ways to control exposure in devices havingwide angle lenses.

SUMMARY OF THE INVENTION

It is an object of the invention to provide improved control of theexposure of images from endoscopes or other imaging devices (e.g.,exoscopes, borescopes), through counting pixels in a distorted imagethat are above or below a threshold for exposure control such that theeffects of the unwarping are included in the exposure control. This hasthe advantage of improving image quality over prior systems that controlexposure with the distorted or warped image data. It is another objectof the invention to provide faster feedback control of imaging devicesto enable real time feedback control of video signals, and to provideefficient distortion models for achieving the same for one or multipleregions of interest viewable by a device. This has the advantage ofshowing smooth, clear video and improving various feedback controlissues that result slow exposure control.

According to a first aspect of the invention, an exposure controlprocess is used that, within a selected region of interest, applies apredetermined weight to each pixel location of the distorted image. Thepredetermined weight for a pixel location may depend on the number oftimes the pixel value at that location is used to produce a pixel in theundistorted image. The weight for a pixel location also may depend onthe weighting applied to each instance a pixel is used to produce apixel in the undistorted image. The weights are used to scale the pixelintensity values (e.g., pixel luminance values, pixel color values,and/or other lightness intensity controlling values) for all pixellocations of interest, which are then summed to produce a pixel countthat is representative estimate of the pixel counts as if they had beencounted in the unwarped image. In some versions, the region of interestand viewing angle may be changed by a selection corresponding to asub-set of pixels available from the image sensor.

According to a second aspect of the invention, a solid-state look-aroundendoscope is provided that includes an improved exposure control systememploying the first aspect of the invention. The viewing angle of thedevice may be changed by selection of a sub-set of pixels of the imagesensor that corresponds to the desired region of interest. Further, theinvention's techniques can be generally applied to an automatic exposurecontrol technique for warped images (e.g., images captured with afish-eye lens) produced from other devices.

In a third aspect, an imaging scope system is provided including a lightemitting element, an optical assembly including a wide angle lenselement, and an image sensor assembly including an image sensorconfigured to receive at least a portion of light focused through theoptical assembly and produce output signals. Image forming circuitryreceives the output signal and produces an image signal, but the imagesignal includes distortion resulting from the wide angle lens element.Exposure control circuitry is coupled to receive the image signal fromthe image forming circuitry, and coupled to the light emitting element,the image sensor, and the optical assembly to perform feedback control.The exposure control circuitry models an inverse effect of thedistortion by (i) estimating pixel weights that respective pixels of theimage signal including distortion have when producing an undistortedversion of the image signal and (ii) scaling pixel intensity values byrespective pixel weights to produce a weighted value for the pixels. Theweighted pixel values are provided to an exposure control pixel countingprocess to produce one or more exposure feedback control signals. Theseare transmitted to the device's light emitting element, the opticalassembly, or the image sensor assembly. The system also has imageprocessing circuitry configured to receive the image signal includingdistortion and produce an undistorted version of the image signal.

In some implementations of any of the above aspects of the invention,the exposure control circuitry is configured to operate in parallel tothe image processing circuitry. The exposure control circuitry may beconfigured to provide the weighted pixel values to the exposure controlpixel counting process as part of a digital signal processing sequencebefore the image processing circuitry produces the undistorted versionof the image signal. In some versions, the exposure feedback controlsignals take effect on the imaging device for the subsequent image framefrom which the exposure levels are calculated. The image formingcircuitry may include image selecting circuitry that receives the imagesignal and produces a region of interest signal that corresponds to aregion of interest that is less than the entire image area carried inthe image signal. In some versions, the region of interest and viewingangle may be changed by a selection corresponding to a sub-set of pixelsavailable from the image sensor. The exposure control circuitry may beformed in a field programmable gate array (FPGA) or a graphicsprocessing unit (GPU), or may be split between or among processingdevices.

When estimating pixel weights, the circuitry may evaluate firstcoefficients which describe curves from which the pixel weight estimatesare calculated using the respective pixel's coordinates. These firstcoefficients may include cubic spline coefficients or quadraticpolynomial coefficients. The use of such coefficients allows the storageof an accurate pixel weight model with little memory compared to storingthe pixel weights themselves. The first coefficients may be produced byevaluating second coefficients of second curves which describe thevalues of the first coefficients as a function of a viewing angle of thedevice.

According to a forth aspect of the invention, a method is provided tocontrol the image exposure on an imaging device. The method includesreceiving a series of sequential image frames including multiple pixelsfrom an imaging device having a wide angle lens that producesdistortion. The method activates a first distortion model for estimatinga pixel weight that the distorted image pixels would have in anundistorted version of an image frame. To control exposure of theimaging device, the method uses the required pixels, and scales eachpixel intensity value by a respective pixel weight from the firstdistortion model to produce a weighted pixel value for each pixel. Thesevalues are fed to an exposure control pixel counting process to produceone or more exposure level signals. From these signals, the methodcreates exposure feedback control signals that are transmitted to therelevant parts of the imaging device in such a manner that they controlexposure features of the imaging device. The feedback signals maycontrol the integration time on an imaging array sensor of the imagingdevice, or the brightness of a light source of the imaging device, theaperture of the imaging device, or other exposure related functions. Insome versions, the exposure feedback control signals take effect on theimaging device for the subsequent image frame from which the exposurelevels are calculated. The method also produces an undistorted versionof the image frame, preferably independently of the exposure controlsteps.

In some implementations of the forth aspect, in response toinstructions, the method adjusts the image devices angle of operationand activates a second distortion model for performing exposure controlat the new angle of operation, and starts working with a stream of imageframes produced at the new angle. In some versions, the region ofinterest and viewing angle may be changed by a selection correspondingto a sub-set of pixels available from the image sensor. A device maystore a distortion model for each angle of view which the device may beconfigured to observe. The distortion models may be stored moreefficiently by storing coefficients of curves that describe the models.

The process of activating a distortion model may include evaluatingfirst coefficients which describe curves from which the weight of arespective pixel is calculated using the respective pixel's coordinates.The first coefficients may include cubic spline coefficients andquadratic polynomial coefficients. The first coefficients may beproduced by evaluating second coefficients of second curves whichdescribe the values of the first coefficients as a function of a viewingangle of the device.

In some implementations of the forth aspect, the imaging device is amedical scope (e.g., an endoscope or exoscope) configured to allow auser to select from multiple regions of interest viewing at multiplerespective angles of view, with the method further including storing arespective distortion model for each of the regions of interest. Otherversions may work with other types of imaging devices.

Implementations of the invention may also be embodied as software orfirmware, stored in a suitable medium, and executable to perform variousversions of the methods herein. These and other features of theinvention will be apparent from the following description of thepreferred embodiments, considered along with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is cross-section of the distal end of a prior art wide-angleendoscope in which the invention may be implemented.

FIG. 1B is a depiction of the image sensor of the same prior artendoscope relative to the endoscopic field of view.

FIG. 2A is a hardware block diagram of an example image capture deviceaccording to an example embodiment of the invention.

FIG. 2B is a block diagram showing more detail ROI production circuit ofFIG. 2A.

FIG. 2C is a functional block diagram showing more detail of an exampleprocessing circuit of FIG. 2A.

FIG. 2D is a hardware block diagram of another example image capturedevice with separate exposure processing circuitry.

FIG. 3 is a flowchart of an exposure control process.

FIG. 4 is a flowchart for an example process of constructing andproviding a distortion model.

FIG. 5 is a flowchart of a process for loading and activating adistortion model for a selected ROI.

FIG. 6 shows an example area-stretch weight surface plotted over animage device area.

FIGS. 7A-B show two example curves of binned area-stretch weights.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

The invention provides improved control of exposure for imaging deviceswith a wide angle lens system, such as an endoscope device that gathersan endoscopic image field at least spanning the longitudinal axisrelatively large angles off of the longitudinal axis. A preferredversion is employed in an endoscope such as the one described in U.S.Pat. No. 8,814,782 to Hale, et al, issued Aug. 26, 2014, which is herebyincorporated by reference. The techniques and features herein may alsobe embodied in other types of image capture devices, digitalmicroscopes, certain digital cameras, virtual reality (VR) cameras andVR camera rigs of multiple cameras, mobile phones equipped with imagingsub-systems, and automotive vehicles equipped with imaging sub-systems,for example.

FIG. 1A shows a cross section of a distal tip 100 of a prior artwide-angle endoscope in which the invention may be implemented. Thedepicted distal tip 100 has longitudinal axis 6, a viewing window 70,and an optical assembly including a wide angle lens system 165 withoptical center 160 and a transmission system 150. FIG. 1B is a diagramdepicting an image sensor of the same prior art endoscope relative tothe endoscopic field of view. As shown in FIG. 1B, in some versions, theregion of interest and viewing angle may be changed by a selectioncorresponding to a sub-set of pixels available from the image sensor.The same scheme may be employed in other devices that are notendoscopes. Referring to both Figures, the optical center 160 isangularly offset from the longitudinal axis 6 of the endoscope 100 andcovers a viewing range 120 of 160 degrees from −45 to +115 degreesrelative to the longitudinal axis. From this configuration, the wideangle lens system 165 simultaneously gathers an endoscopic image field130 that spans the longitudinal axis and an angle greater than ninetydegrees to the longitudinal axis. As a result, the simultaneous imagefield gathered by the endoscope provides both forward and retrogradeimaging. Providing a variable view endoscope that spans this range isbeneficial because it enables a user to view objects that reside infront of the endoscope and behind the standard fields of view forendoscopes. This improves the ability of a user to safely operate andhandle the device in the body cavity. Further by incorporating a wideangle lens with an optical center that is angularly offset relative tothe longitudinal axis, the endoscope can more accurately mimic theviewing capabilities and function of a fixed angle endoscope.

The image field gathered by wide angle lens system 165 is conveyed totransmission system 150, which in turn conveys the wide angle field ofview to an image sensor surface area 170 that comprises image surfacearea 170 formed by a plurality of pixels that gather light images andconvert the images to output signals. The image surface area 170 ispreferably rectangularly shaped with a longitudinal dimension that isgreater than its lateral dimension, but can also be a variety ofdifferent shapes, such as square, circular or oval. Also, it ispreferable that the image surface area 170 has an HD aspect ratio of16:9. Since a wide-angle lens system can provide uneven informationdistribution, without correction an HD image sensor enables the crowdedinformation regions to be captured and displayed on a monitor. As shownin FIG. 1B, image surface area 170 partially captures field 130. It ispreferable that the longitudinal dimension of image surface area 170substantially correspond to the entire longitudinal dimension of field130. This enables the endoscopic system to provide the user with animage or a range of regions of interest that span the field of view ofthe endoscope. However, image surface area 170 only captures a portionof the lateral dimension of field 130. The lateral dimension of area 170can be chosen such that the distortion of an image laterally is minimaland not detected by the human eye. Further, by limiting the lateraldimension of the sensor, the cross-sectional area of the endoscope canbe more efficiently used. For instance, the lateral dimension of thewide angle lens can be reduced and consequently reduce the overall sizeof the endoscope. Also, the area of the field of view not captured bythe sensor can be used carry a fiber optic illumination system.

FIG. 1B also depicts specific regions of interest (ROIs) at 0, 30, 45and 70 degrees which can be selected by a user over a designated range190. A region of interest is an image area formed on the image surfacearea that is a subset of the overall field of view captured by thesensor. The center of the area of the ROI corresponds to a selectedviewing angle chosen by a user, in this case a longitudinal viewingangle, but other offset directions may be used. The overall area of theROI can correspond to the field of view typically provided by a fixedangled endoscope for that same angle. Alternatively, the overall area ofthe ROI can be chosen to provide a minimal distortion variation acrossthe overall area. Still further, the overall area of the ROI can bechosen such that the field encompassed by a viewing angle at leastpartially overlaps with an adjacent standard viewing angle, such as 30and 45 degrees. ROIs that are sized to overlap with adjacent viewingangles assist a user in maintaining orientation in the event that aviewing angle is changed.

Because digital cameras and scopes employing imaging devices and relatedcircuitry for signal capture, processing, correction, and exposurecontrol are well-known, the present description will be directed inparticular to elements forming part of, or cooperating more directlywith, methods, and apparatus and program products in accordance withexample embodiments of the invention. Elements not specifically shown ordescribed herein are selected from those known in the art.

FIG. 2A is a hardware block diagram of an example image capture deviceaccording to an example embodiment of the invention. While a diagram isshown for a medical scope and its associated display system, theinvention is applicable to more than one type of device enabled forimage capture, such as endoscopes incorporating solid state imagers,digital microscopes, digital cameras, VR camera rigs, mobile phonesequipped with imaging sub-systems, and automotive vehicles equipped withimaging sub-systems, for example. A light source 8 illuminates subjectscene 9 and light 10 reflected from (or, alternatively, transmitted byor through) the subject scene is input to an optical assembly 11, wherethe light is focused to form an image on solid-state image sensor 20.The light source may take any suitable form of light emitting element,including a fiber optic light transmission assembly to which an internalor external light source may be connected. Optical assembly 11 includesat least one wide-angle lens element, and preferably a wide angle lenssystem such as system 165 of FIG. 1, such that optical assembly 11focuses light which represents a wide field of view. Image sensor 20converts the incident light to an electrical signal by integratingcharge for each picture element (pixel). The image sensor 20 of thepreferred embodiment is an active pixel complementary metal oxidesemiconductor sensor (CMOS APS) or a charge-coupled device (CCD). Thetotal amount of light 10 reaching the image sensor 20 is regulated bythe light source 8 intensity, the optical assembly 11 aperture, and thetime for which the image sensor 20 integrates charge. An exposurecontroller 40 responds to the amount of light available in the scenegiven the intensity and spatial distribution of digitized signalscorresponding to the intensity and spatial distribution of the imagefocused on image sensor 20. To perform fast and accurate control,exposure controller 40 uses exposure level signal 302 (FIG. 2C)calculated from the warped or distorted ROI pixel data 27 received atsignal processor 30, as further described below.

In use, an analog signal from the image sensor 20 is processed by analogsignal processor 22 and applied to analog-to-digital (A/D) converter 24for digitizing the analog sensor signals. ROI (region of interest)production circuit 25 produces warped ROI signals 27 by identifying andselecting, as directed by system controller 50, a region of interestwithin the captured image. The warped ROI signals 27 describe a seriesof images frames that are warped, i.e. spatially distorted, as an effectof the optical assembly 11 using a wide-angle lens. The resulting warpedROI signals 27 are transmitted to the image processing circuit 30. It isnoted that while this embodiment of the invention employs a ROIselection from a larger array of available image data, other versionsmay use the entire field of data received from image sensor 20. Timinggenerator 26 produces various clocking signals to select rows and pixelsand synchronizes the operation of image sensor 20, analog signalprocessor 22, and A/D converter 24. Image sensor stage 28 includes theimage sensor 20, the analog signal processor 22, the A/D converter 24,and the timing generator 26. The functional elements of the image sensorstage 28 can be fabricated as a single integrated circuit as is commonlydone with CMOS image sensors or they can be separately-fabricatedintegrated circuits.

Image processing circuit 30 is one of three programmable logic devices,processors, or controllers in this embodiment, in addition to a systemcontroller 50 and the exposure controller 40. Although this distributionof imaging device functional control among multiple programmable logicdevices, programmable logic devices, and controllers is typical, theseprogrammable logic devices, processors, or controllers can be combinablein various ways without affecting the functional operation of theimaging device and the application of the invention. These programmablelogic devices, processors, or controllers can comprise one or moreprogrammable logic devices, digital signal processor devices,microcontrollers, or other digital logic circuits. Although acombination of such programmable logic devices, processors, orcontrollers has been described, it should be apparent that oneprogrammable logic device, digital signal processor, microcontroller, orother digital logic circuit can be designated to perform all of theneeded functions. All of these variations can perform the same functionand fall within the scope of this invention.

In the illustrated embodiment, image processing circuit 30 manipulatesthe digital image data according to processes that are either programmedinto the circuit (in the case of programmable logic devices) or loadedinto the circuit program memory as programming instructions (in the caseof processors and controllers such as a graphics processing unit (GPU)).The digital image data manipulation includes, but is not limited to,image processing steps such as color filter array demosaicing, noisereduction, color correction, and gamma correction. In this version thedigital image data manipulation performed by image processing circuit 30also includes and calculating the exposure level signals 302 (FIG. 2C)and unwarping the image data within the selected ROI to produce finalunwarped image signals 32.

The system controller 50 controls the overall operation of the imagecapture device based on a software program stored in program memory 54.This memory can also be used to store user setting selections and otherdata to be preserved when the camera is turned off. System controller 50controls the sequence of image capture by directing exposure controller40 to set the light source 8 intensity, the optical assembly 11aperture, and the time for which image sensor 20 integrates charge(integration time), directing the timing generator 26 to operate theimage sensor 20 and associated elements, and directing image processingcircuit 30 to process the captured image data. A data bus 52 includes apathway for address, data, and control signals, and connects systemcontroller 50 to image processing circuit 30, program memory 54, systemmemory 56, timing generator 26 and other related devices. Exposurecontroller 40 may also be connected to data bus 52, or may have aseparate bus or signal lines connecting to system controller 50.Exposure controller 40 is configured to control exposure features of theimaging device using one or more exposure feedback control signals. Inthis version, three exposure feedback control signals are shown, a lightsource control signal 43, an aperture control signal 45, and anintegration time control signal 47 (which is shown in dotted linesbecause it is typically not a direct signal path, but is implementedthrough system controller 50 indirectly to control image sensor 20 suchas by controlling the timing generator 26). These signals respectivelyallow the exposure controller to set the exposure light source 8intensity, the optical assembly 11 aperture, and the time for which theimage sensor 20 integrates charge. While three exposure feedback controlsignals are shown in this embodiment, other versions may use any one ortwo of these, or may include additional signals to control exposurecharacteristics of the device, such as a gain value amplifying one ormore signal values from the image array, for example.

After the image processing is complete, the processed image data iscontinuously sent to video encoder 80 to produce a video signal. Thissignal is processed by display controller 82 and presented on imagedisplay 88. This display is typically a liquid crystal display backlitwith light-emitting diodes (LED LCD), although other types of displaysare used as well. The processed image data can also be stored in systemmemory 56 or other internal or external memory device.

The user interface 60, including all or any combination of image display88, user inputs 64, and status display 62, is controlled by acombination of software programs executed on system controller 50. Userinputs typically include some combination of typing keyboards, computerpointing devices, buttons, rocker switches, joysticks, rotary dials, ortouch screens. The system controller 50 manages the graphical userinterface (GUI) presented on one or more of the displays (e.g. on imagedisplay 88). The GUI typically includes menus for making various optionselections.

FIG. 2B shows in more detail the ROI production circuit 25 from FIG. 2A.When a user selects a region of interest viewing angle through userinputs 64 of user interface 60, the system controller 50 transmits viadata bus 52 a region of interest field selection that is received byimage control circuitry 204. Image control circuitry 204 in turnproduces a field control signal used to identify a portion of thecaptured image, the ROI, associated with the user selection. Imageselecting circuitry receives the field control signal to identify andselect the portion of the captured image to produce warped ROI signals27 containing the image data for the series of warped image frames inaccordance to the region of interest viewing angle selected by the user.In some versions the ROI is a square or rectangular region and thesignals produced from ROI production circuit 25 contain images with thedesired ROI boundaries, while in other versions the ROI may be circular(a traditional view used in medical scope devices, for example), andwarped ROI signals 27 may contain rectangular warped images that requirefurther masking of areas to produce a final unwarped image signals 32.

FIG. 2C shows in more detail the example image processing circuit 30from FIG. 2A, which processes the image data of each frame to producethe desired form of image from the data received. Image processingcircuit 30 generally performs several image processing steps, reflectedin the depicted sequence of in which there are N number of imageprocessing steps. Zero or more image processing steps precede thecalculate exposure level signals step 300, which is the Lth step andproduces exposure level signals 302. Zero or more steps may follow thecalculate exposure control signals step 300 before performing the imageunwarping step 304, which produces unwarped image signals 306. Zero ormore steps follow the image unwarping step 304 to produce the finalunwarped image signals 32. Data bus 52 transports the exposure levelsignals 302 to the system controller, or directly to the exposurecontroller 40, and transfers other information both ways between thesystem controller 50 and each of the image processing steps. Thetransported information typically includes image processing stepparameters and user-selected option indicators.

While FIG. 2C shows versions of the invention in which the imagingprocessing steps are all performed consecutively (i.e. in series)following a single data path, other variations are possible wherein atleast two of the image processing steps are performed in parallelfollowing two or more data paths and that perform the same function asdescribed and shown in FIG. 2C. It is noted that the step of producingunwarped ROI signals 306 does not require output from the step ofcalculating exposure level signals 300, and so these processes may beperformed in parallel by different processing cores, different threads,or different processors or digital logic, and the result of the exposurelevel signal calculation 300 be passed to exposure controller 40. Theexposure level signal calculation 300 is preferably conductedindependently of the unwarped ROI signal calculation 306, and block 300may be conducted before or in parallel with block 306. All of thosevariations fall within the scope of the invention.

The depicted loop of calculating exposure level signals 302 from thewarped image data in signals 27, and using the exposure level signals togenerate exposure control feedback signals 43, 45, and 47, represents areal-time control loop that automatically adjusts the exposure level nomatter what viewing angle of the device. In one embodiment, thereal-time control loop accepts a distortion variable or area stretchweight for each pixel that reflects a weighted contribution ofcaptured-imaged pixels within a region of interest (ROI) of the warpedimage. The pixels are weighted according to how much the pixel is“stretched” when later providing an undistorted image to display, save,and/or analyze. The area stretch weights are preferably generated with adistortion model, as further described below, but may be stored directlyor otherwise calculated based on known distortion characteristics of thedevice's optical assembly.

The depicted real-time exposure control loop requires quick, butaccurate processing. The most intensive part of the processing toproduce the exposure level signals is pixel counting, wherein pixelswith a digital value above or below a threshold are counted. For warpedimages, the relative contribution of the pixel, as projected on anun-warped or undistorted image, should be accounted for since thecapture imaged undergoes unwarping/stretching algorithms prior todisplay or storage. Pixel counting can be done on a processed (unwarped) image, but unwarping typically takes too long for real-timeneeds. The techniques herein incorporate the unwarping effects in thepixel counting process. For example, a predetermined area-stretch weightof pixel location may depend on the number of times the pixel value at alocation is used to produce a pixel in the stretched/undistorted image.The area-stretch weight may be based on an optic model that accounts forFOV (field of view), actual viewing angle, and distortion characteristicfunction of the wide angle lens and the remaining optical assembly (theother lenses and optical elements through which the image light passesbefore hitting the sensor). The area-stretch weights can either bestored in memory blocks or calculated by a subsystem, which usesknowledge of the unwarping algorithm as further described below. Becausethe pixel-counting is done within an automatic real-time exposurecontrol loop, the counting must be done quickly enough such that thereis no substantial delay in producing a real-time control signal.Real-time in this context typically means that the counting is donequickly enough to keep up with the frame rate of the system, so that theexposure control feedback controls signals may be generated based oneach image. Depending on the timing of the image sensor, this may allowthe exposure control feedback signals to be received and the newsettings take effect before the next frame of data is acquired from theimaging sensor (a blanking interval often leaves time between frames ofacquired data). In such cases, real-time feedback control provides thatthe effects of the feedback are seen on the very next frame. In somecases, the feedback control signals will take effect two frames afterthe frame from which the exposure level signals are calculated.Real-time may also mean that the pixel counting and exposure feedbackcontrol signal generation is done before the current image frame hasfinished its image processing, the remaining processing steps in FIG.2C, for example. Since unwarping algorithms are relatively slow, it isdifficult to first unwarp the distorted image and then do thepixel-counting to produce the control signal in the time allotted. It istherefore advantageous to count pixels of the distorted image in a waythat the counting includes the effects of the unwarping, such as withthe area-stretch weights used in the preferred version herein.

FIG. 2D is a hardware block diagram of an image capture device accordingto another embodiment of the invention with a different hardwareconfiguration. Generally, the depicted elements are similar to those ofFIG. 2A, but the calculation of exposure level signals 302 is performedin exposure processing circuitry 41 separate from the image processingcircuitry 30, such as in a separate signal processor or a digital logicdevice such as an FPGA. Exposure processing circuit 41 may also beintegrated with exposure controller 40, for example in an FPGA or DSP.As shown, exposure processing circuitry 41 is separate from exposurecontroller 40 and connected to exposure controller 40 and systemcontroller 50 via system bus 52. In this version, system controller 50will typically send commands and model data to exposure processingcircuit 41 to configure the circuit with an appropriate distortion modeland resulting area stretch weights for the ROI currently selected by theuser, as will be further discussed below with regard to the distortionmodels employed in some embodiments.

Importantly, in each of the example embodiments of the invention herein,the calculation of exposure level signals at depicted step 300 does notrequire the unwarped image signals 306 as its input. This allows animproved exposure control that accounts for the warping that mightotherwise skew the exposure control process, yet still provides a fastenough signal that the exposure controller can perform real-timefeedback control of the image capture device. Preferably, while datafrom the current image frame is being processed, the exposure feedbackcontrol signals are generated based on the current image frame areoutput to the imaging device. However, in some versions, the pixelcounting process, and signal generation at the exposure controller 40,may cause delay in the control process that a delay of more than oneframe is present before exposure feedback control signals 43, 45, and 47are received at the imaging device. In such cases, the delay is stillmuch shorter than that resulting from using the unwarped image data toperform exposure control, with resulting improvements in speed andstability of the exposure control.

FIG. 3 is a flowchart of an exposure control process according to someembodiments, which may be employed with the example hardware andfunctional designs of FIGS. 2A-D, or may be employed with other hardwaredesignated with a similar purpose. Generally the depicted process usesmodels of the distortion or warping produced by the wide angle lens inthe imaging device, referred to as distortion models, which characterizethe weight that each pixel in the ROI would have when producing anundistorted image, but without producing the image to calculate theweight. These models may be prepared in advance by the system designers,or may be calculated automatically through observation andcharacterization of the lens and optical system used, or the distortionpresent in the image data (such as the warped ROI signals 27) obtainedfrom the device.

In any case, the depicted process begins at block 301 with providing amodel of the distortion for each ROI for which the device may beconfigured to produce images. The process of constructing and providingsuch models is discussed further with respect to FIG. 4. When theinvention is embodied in a solid-state look-around endoscope device,such as the device described in the incorporated U.S. Pat. No.8,814,782, a distortion model is provided for each ROI which the deviceis capable of producing. In some versions, a device is capable ofcontinuous movement within a range of movement, providing a large numberof possible ROIs. In such case, while a model may be provided for eachdegree or fractional degree of movement, it is generally better toprovide a model generator configured to automatically create theappropriate distortion models. Further, while the use of calculateddistortion models to provide the desired pixel weights is preferred forspeed of loading the models from system memory, other versions may storethe distortion model as the raw pixel weights that characterize thedistortion.

A new region of interest may be selected at step 303 or the process maybegin with a default or last-used region of interest selection.Generally, the process of the user selecting a region of interest andthe device producing the view of the selected region is described inU.S. Pat. No. 8,814,782, and will not be further described here to avoidrepeating information unnecessarily. The process may include selectingfrom preset views, or receiving a panning command to move the view orotherwise choose a new angle at which the imaging device will operateand, in response, adjusting the image devices angle of operation andactivating or loading from memory (block 305) a distortion model forperforming exposure control at the new angle of operation. The exposurecontrol process is typically implemented under control of softwarestored in system memory 56 and run on system controller 50 and imageprocessing circuit 30. Some functions may be implemented in software orfirmware of exposure controller 40 and exposure processing circuit 41.Note that the distortion models as referred to herein are separate andindependent from the processes at block 315 used to unwarp the imagedata and produce unwarped image signals 32. At block 305, the imageprocessing device and the exposure control circuitry start receiving astream of distorted image frames produced at the newly selected ROI.

Next at block 307, the process begins processing a first frame or imagefrom the stream of images. The processing of a frame is preferablycompleted before a new frame is received, allowing the depicted processloop to update the exposure control signals with each frame. The loop ispreferably performed by exposure control program code and imageprocessing program code executed on one or more devices as describedabove. The process at block 309, applies the distortion model todistorted pixel data to produce weighted pixel value for each pixel inselected ROI. This block, in the preferred version, multiplies the pixelintensity value, or other values controlling the lightness or perceivedbrightness of the pixel depending on what encoding the imagingprocessing system employs (such as the YCbCr and Y′CbCr color), for eachpixel with the weight assigned to that pixel in the distortion model.That is, for each of a group of designated pixels defining a region ofinterest, the process scales a pixel intensity value by a respectivepixel weight from the distortion model to produce a weighted pixel valuefor each respective pixel. It is noted that the pixels outside the ROIare not used. They may be zeroed out, or simply excluded from thecalculation. Because the pixels actually used for the ROI vary for eachROI, some definition of the boundary of the pixels employed for each ROIis needed. The exclusion of pixels outside the ROI may be done beforethe application of the distortion model at block 309, or may be doneconcurrently. In some versions, the region of interest is circular,requiring an elliptical set of pixels from the distorted image datacontained in warped ROI signals 27. Such an ellipse may be defined byits center and axes lengths, and then multiple ROI's of a particulardevice may be defined by coefficients describing a curve of the two axeslengths versus angle of view. In this manner, the ROI boundaries may bestored and retrieved without storing an entire mask or boundarycoordinates for each ROI.

The process next goes to block 311 where the weighted pixel values arefed to a pixel counting process to produce one or more exposure levelsignals. Block 311 may be performed by dedicated circuitry such asexposure processing circuit 41 (FIG. 2D), or by the image processor 30or the exposure controller 40 executing exposure control program code. Asingle exposure level 302 signal may be calculated by counting thepixels whose weighted pixel value is above a designated threshold. Insome versions, multiple exposure level signals 302 may be calculated.For example, exposure level signals may be calculated indicating aspatial distribution of the brightness corresponding to the ROI, orexposure level signals may be calculated for sub-regions of the ROI.Further, exposure level signals may be calculated based on identifyingone or more features in the ROI and calculating levels corresponding tothe brightness of those features separate from the overall ROI exposurepixel counting. In such versions, the process may require dedicatedhardware in the form of exposure processing circuit 41.

The process at block 312 then, based on the exposure level signals 302,generates one or more exposure feedback control signals. Block 312 istypically performed by exposure control program code running on exposurecontroller 40, but these functions may be integrated with othercontroller circuits such as system controller 50, for example. In thesimplest forms, block signals to increase or decrease the brightness ofthe device light if the exposure level signal is under or over a desiredthreshold, or adjust the image sensor integration time accordingly. Theaperture may also be adjusted. The exposure feedback control signals mayalso be based on other variables such as the distance to the subjectscene. The feedback control signals may be generated entirely atexposure controller 40, or some of the calculations or comparisonsneeded to generate the exposure feedback control signals from theexposure level signals may be performed by the device that calculatesthe exposure level signals (exposure processing circuitry 41 or imageprocessing circuit 30), and then transmitted to exposure controller 40.

Next, at block 313, the process transmits the one or more exposurefeedback control signals to the imaging device in such a manner thatthey control exposure features of the imaging device for subsequentframes in the series of sequential image frames. These signals willtypically include light source control signal 43, aperture controlsignal 45, and integration time control signal 47. Other signals mayalso be generated based on the exposure level signals to control otherfeatures on the imaging device.

Next at block 315, the process producing an undistorted version of theimage frame, the unwarped ROI signals 306 (FIG. 2C) typically usingknown methods of linear interpolation based on the specific warpingresulting from the system optics. This block is typically done in theimage processor 30 by executing image processing program code. Next atblock 317, further image processing steps may be performed as fordisplay and output of the image having the desired view.

FIG. 4 is a flowchart for an example process of constructing andproviding a distortion model according to an example embodiment.Generally the models are created with the following considerations.

Area-Stretch Weighting

Since the pixels outside of the ROI are not used to form the displayedimage, it is better to exclude them from the exposure controlcalculations. For the pixels inside the ROI, some of them will be usedmore frequently than others during the interpolation of the displayedimage. This is because a small area in the sensor image may get unwarpedinto a relatively larger displayed image area than another small area onthe sensor image. Therefore, the distortion models a weight for eachpixel accounting for ROI pixels that get “stretched out” more thanothers that get “stretched out” less. An example group of area stretchweights for an example ROI is shown in the 3D graph of FIG. 6, whichshows the vertical axis as the area stretch weight value for a pixel,and the two horizontal axes as the X-Y locations of pixels. As can beseen in the graph, the elliptical ROI depicted is toward one end of therange of movement of the device, with the left-side depicted edge pixelsgiven more weight in the exposure control because they contribute tomore pixels in the undistorted image.

Exposure Control Area-Stretch Weight Model

Bilinear interpolation of the sensor image can be used to produce theunwarped displayed image data based on the known distortion from thesystem optics. Other image interpolation method can also be used. If,for all sensor pixels in the ROI, every time a sensor pixel is used inthe interpolation, the pixel's interpolation weight is accumulated, theinterpolation process provides a measure of how much area each sensorpixel covers in the display image, quantified as an area-stretch weight.This procedure will generate a 2-dimensional surface in a 3-dimensionalspace with x-coordinate, y-coordinate, and area-stretch weight as thethree axes. For a given set of optics (i.e. FOV, actual viewing angle,and distortion characteristic function), one such 2-D surface can becreated for each angle of view (see FIG. 6). Given that it may bedesired to have many available viewing angles (e.g. 0 degrees to 60degrees in steps of 0.1 degree), it is generally much less burdensome touse area-stretch weights only if there is a simple model to generatethem (the alternative is to store an entire frame's worth of weights foreach available angle of view). Fortunately, the radial distortionsymmetry of the optics allows for creation of a suitable distortionmodels.

Referring to the flowchart of FIG. 4, an example process is given toprovide distortion models (block 301) for each ROI provided by thedevice. To create the distortion models, a number N angles of view areselected (e.g. 0 degrees to 30 degrees in steps of 1 degree) and thecorresponding area-stretch weights are calculated at block 402,resulting in weights for each pixel, which may be considered as asurface for each ROI when graphed with pixel x-y locations as in theexample weight surface of FIG. 6. The area-stretch weight surfaces canhave fine structures, depending on how the sensor pixel lattice lines upwith the new pixel lattice within a small area of the new, unwarped,image. It can therefore be understood after appreciating this disclosurethat there are many possible ways to create distortion models, some ofwhich may be better for differing types of optics and image sensorcombinations. The process described here is only one example whichprovided good results for an SSLA endoscope embodiment as describedherein.

To produce a simple model from the weight surfaces, the process at block403 first smoothes the area-stretch weights by filtering, in this caseusing a minimum filter with a 7×7 window followed by an 11×11 boxcarfilter. For area-stretch weight surfaces that do not exhibit finestructures on the weight surfaces, such filtering may not be required.

Next the process at block 404 partitions the data for each surfaceaccording to the polar angle phi, into 288 bins. This number will, ofcourse, tend to vary and may be higher as higher resolution devices arecreated giving many more pixels in each bin at the outer edge of thesurface. To find phi at each pixel location, the process uses phi=a tan2(x−X/2, y−Y/2). That is, the process shifts the coordinate system fromthe upper left to the center of the ROI. Bin the weights into theappropriate phi bin, keeping track of the associated radii. To find theradius r at each pixel location, use r=sqrt((x−X/2)̂2+(y−Y/2)̂2). Thereshould be now (288 bins/angle of view)*(N angles of view) weight curvesas functions of radius.

Using the weight curves, the process at block 405 now finds coefficientsto describe the weight curve. In this version, the process fits a seriesof cubic splines or quadratic polynomials to the weight curve for eachviewing angle. Of course, other suitable functions may be used toproduce coefficients describing the weight curves. The selection of howmany splines, polynomials, or other functions to use will vary dependingon device. For this version, the curves fitting is done as follows:

For each angle of view, the process picks the phi bin that has thelargest domain in r. FIGS. 7A-B show two example curves of binnedarea-stretch weights. The bin on the right has a larger domain than thebin on the left, but there is good agreement between them for thecommon-domain portion. Each bin has some common domain with thislargest-domain bin, and they have good agreement in the common domain.Next the process finds the knee of each weight curve, which is theradius value where the slope suddenly increases (the data tends to“bunch up” there). This can be done by taking the derivative of thecurve and checking for a maximum above a specified value, for example.On the curve of FIG. 7B the knee can be seen to be about r=700. If thereis no knee, the process finds the endpoint of the good data. On thecurve of FIG. 7A, there is no knee and the endpoint of the good data isabout 600 (the sharp drop-off that starts around 600 is just an edgeartifact due to the edge of the interpolation area).

Next, the curves are to create a first set of coefficients, which may bereferred to as distortion curve coefficients, using these steps: Fit acubic spline (rCi) with three knots to each weight curve below the curveknee (or endpoint if there is no knee). The first knot should be at zeroradius, the third knot will either be at the knee if it exists or at theendpoint. Fit a quadratic polynomial (rQi) to the data above the knee.Constrain the quadratic to have a value equal to the cubic spline fit atthe knee. If there is no knee, there is no need to fit the quadraticpolynomial. This is done because a good fit cannot be obtained with asingle spline without significantly increasing the number of knots orincreasing the order of the spline or both. By splitting the data intotwo segments, below and above the knee, two good and simple fits can beobtained. A potential drawback of fitting in this way is that thegradient is not guaranteed to be smooth across the knee point. This isnot expected to cause any problems because the fitted result is notapplied to a displayed image (in which case gradient jumps cause visualartifacts). While splines and quadratic polynomials are used in thisversion, other functions may be to fit the curves.

Beginning at block 406, the process uses the results of the curvesplitting to create a model of the curves that can be applied acrossdifferent viewing angles. Now there should be (4 parameters per cubicspline segment)*(2 cubic spline segments)+(3 parameters per quadraticsegment)*(1 quadratic segment) different parameters, each with N values.These parameters (both coefficients and knot and knee values) arearranged as functions of the N angles of view at block 406. If the kneesand the endpoints are well-chosen, the resulting curves should berelatively well-behaved. Block 406 also assembles all the spline knotsas functions of viewing angle, and assembles all the knee values as afunction of viewing angle.

The process then takes these functions, and at block 407 fits curves tothese coefficient values to model functions them as a function ofviewing angle. In this version, this block involves fitting a quadraticpolynomial (aQci) to each of the cubic spline coefficients. It also fitsa quadratic polynomial to each of the quadratic polynomial coefficients(aQqi). It fits a linear spline with three knots to the two non-zero rCispline knots (linear splines aL1ci and aL2ci). It fits a linear splinewith three knots (aLk) to the knee data. All of these will be functionsof viewing angle. The resulting second set of coefficients make up themodels that can be stored and later evaluated to provide a distortionmodel for each angle of view. At block 408, the coefficients for aQci,aQqi, aL1ci, aL2ci, and aLk are stored in memory for later retrieval bythe system to conduct exposure control.

Exposure Control Area-Stretch Weight Model Implementation

FIG. 5 is a flowchart of a process for loading and activating adistortion model for a selected ROI, which may use a model producedaccording to the process of FIG. 4 or another suitable model. Theprocess of loading and activating a distortion model for the selectedROI may be conducted once for every image frame processed, in versionswhere an FPGA or other dedicated digital logic arrangement is used, ormay be done each time the ROI is changed, as shown in the process ofFIG. 3. The process generally implements block 305 of loading adistortion model for a selected ROI. At block 501, the process retrievesthe second coefficients from memory, which describe curves the firstcoefficients as a function of viewing angle or viewing position of thedevice. For models produced as described above, this includes retrievingthe coefficients for the two quadratic splines aQci and aQqi frommemory, and retrieving the coefficients for the linear splines aL1ci,aL2ci, and aLk from memory, that is the curves that describe the knotand knee values as well.

Next at block 503, the process evaluates the second coefficients toproduce the desired first coefficients at the current viewing angle,that is, the angle or position of the desired ROI. For the model above,this block includes evaluating splines as functions of the currentviewing angle. The set of results defines the spline rCi and thepolynomial rQi that are needed for the current ROI or image frame. Nextat block 505, the resulting curves are then evaluated at each pixellocation in the desired ROI to provide the modeled area-stretch weight.In the example model above, this block includes evaluating the splinerCi or the polynomial rQi as a function of the radius. This is done oncefor each pixel location in the image. The result at each pixel locationis the area-stretch weight for that pixel location.

Next at block 507, the process may apply a mask to mask-off or set tozero those weights which are not used in the current ROI. This ispreferably done by multiplying area stretch weights by zero if they areoutside the ROI, and by 1 if they are inside. This step sets to zero theweight for pixels outside of the ROI, and is typically used for a curvedor circular ROI. The resulting set of weights is provided as thedistortion model (block 309 of FIG. 3) for the exposure control process.

As used herein the terms “comprising,” “including,” “carrying,” “having”“containing,” “involving,” and the like are to be understood to beopen-ended, that is, to mean including but not limited to. Any use ofordinal terms such as “first,” “second,” “third,” etc., in the claims tomodify a claim element does not by itself connote any priority,precedence, or order of one claim element over another, or the temporalorder in which acts of a method are performed. Rather, unlessspecifically stated otherwise, such ordinal terms are used merely aslabels to distinguish one claim element having a certain name fromanother element having a same name (but for use of the ordinal term).

The foregoing has outlined rather broadly the features and technicaladvantages of the invention in order that the detailed description ofthe invention that follows may be better understood. It should beappreciated by those skilled in the art that the conception and specificembodiments disclosed may be readily utilized as a basis for modifyingor designing other structures for carrying out the same purposes of theinvention. It should also be realized by those skilled in the art thatsuch equivalent constructions do not depart from the scope of theinvention as set forth in the appended claims.

Although the invention and its advantages have been described in detail,it should be understood that various changes, substitutions andalterations can be made herein without departing from the spirit andscope of the invention as defined by the appended claims. Thecombinations of features described herein should not be interpreted tobe limiting, and the features herein may be used in any workingcombination or sub-combination according to the invention. Thisdescription should therefore be interpreted as providing writtensupport, under U.S. patent law and any relevant foreign patent laws, forany working combination or some sub-combination of the features herein.

Moreover, the scope of the present application is not intended to belimited to the particular embodiments of the process, machine,manufacture, composition of matter, means, methods and steps describedin the specification. As one of ordinary skill in the art will readilyappreciate from the disclosure of the invention, processes, machines,manufacture, compositions of matter, means, methods, or steps, presentlyexisting or later to be developed that perform substantially the samefunction or achieve substantially the same result as the correspondingembodiments described herein may be utilized according to the invention.Accordingly, the appended claims are intended to include within theirscope such processes, machines, manufacture, compositions of matter,means, methods, or steps.

1. An imaging scope system, comprising: a light emitting element; anoptical assembly including a wide angle lens element; an image sensorassembly including an image sensor configured to receive at least aportion of light focused through the optical assembly and produce outputsignals; image forming circuitry adapted to receive the output signaland produce an image signal including distortion resulting from the wideangle lens element; exposure control circuitry coupled to receive theimage signal from the image forming circuitry, and controllably coupledto at least one of the light emitting element, the image sensor, and theoptical assembly, and further configured to: (a) model an inverse effectof the distortion by (i) estimating pixel weights that respective pixelsof the image signal including distortion have when producing anundistorted version of the image signal; (ii) scaling pixel intensityvalues by respective pixel weights to produce a weighted pixel value forthe respective pixels; (b) provide the weighted pixel values to anexposure control pixel counting process to produce one or more exposurefeedback control signals; and (c) transmit the one or more exposurefeedback control signals to at least one of the light emitting element,the optical assembly, and the image sensor assembly; and imageprocessing circuitry configured to receive the image signal includingdistortion and produce an undistorted version of the image signal. 2.The system of claim 1 in which the exposure control circuitry isconfigured to operate in parallel to the image processing circuitry. 3.The system of claim 1 in which the exposure control circuitry isconfigured to provide the weighted pixel values to the exposure controlpixel counting process as part of a digital signal processing sequencebefore the image processing circuitry produces the undistorted versionof the image signal.
 4. The system of claim 1 in which the image formingcircuitry further includes image selecting circuitry that receives theimage signal and produces a region of interest signal that correspondsto a region of interest that is less than the image.
 5. The system ofclaim 1 in which the exposure control circuitry is formed in a fieldprogrammable gate array (FPGA).
 6. The system of claim 1 in which theexposure control circuitry comprises a graphics processing unit (GPU).7. The system of claim 1 in which estimating pixel weights furtherincludes evaluating first coefficients which describe curves from whichthe weight of a respective pixel is calculated using the respectivepixel's coordinates.
 8. The system of claim 7, in which the firstcoefficients include cubic spline coefficients and quadratic polynomialcoefficients.
 9. The system of claim 7, in which the first coefficientsare produced by evaluating second coefficients of second curves whichdescribe the values of the first coefficients as a function of a viewingangle of the device.
 10. A method of controlling an imaging device,comprising: (a) receiving a series of sequential image frames includingmultiple pixels from an imaging device having a wide angle lensproducing distortion in the image frames; (b) activating a firstdistortion model for estimating a pixel weight that respectivedesignated pixels have in an undistorted version of an image frame; (c)performing exposure control of the imaging device by: for each of agroup of designated pixels defining a region of interest, scaling apixel value by a respective pixel weight from the first distortion modelto produce a weighted pixel value for each respective pixel; providingthe weighted pixel values to an exposure control pixel counting processto produce one or more exposure level signals; based on the one or moreexposure level signals, generating one or more exposure feedback controlsignals; transmitting the one or more exposure feedback control signalsto the imaging device in such a manner that they control exposurefeatures of the imaging device; (d) producing an undistorted version ofthe image frame.
 11. The method of claim 10 further comprising receivinga new angle at which the imaging device will operate and, in response,adjusting the image devices angle of operation and activating a seconddistortion model for performing exposure control at the new angle ofoperation, and receiving a stream of image frames produced at the newangle.
 12. The method of claim 10 in which at least one of the one tomore exposure feedback control signals controls the integration time onan imaging array sensor of the imaging device.
 13. The method of claim10 in which one of the one or more exposure feedback control signalscontrols the brightness of a light source of the imaging device.
 14. Themethod of claim 10 in which one of the one or more exposure feedbackcontrol signals controls the aperture of the imaging device.
 15. Themethod of claim 10, where in activating a distortion model includesevaluating first coefficients which describe curves from which theweight of a respective pixel is calculated using the respective pixel'scoordinates.
 16. The method of claim 15, in which the first coefficientsinclude cubic spline coefficients and quadratic polynomial coefficients.17. The method of claim 10, in which the performing exposure control ofthe imaging device is done independently of producing the undistortedversions of the image frames.
 18. The method of claim 10, in which theimaging device is a medical scope configured to allow a user to selectfrom multiple regions of interest viewing at multiple respective anglesof view, the method further comprising storing a respective distortionmodel for each of the regions of interest.
 19. Exposure controlcircuitry operable to receive an image signal that includes distortionresulting from a wide angle lens element and operable to control atleast one of a light emitting element, an image sensor, and an opticalassembly including the wide angle lens, the exposure control circuitryfurther configured to: (a) model an inverse effect of the distortion by(i) estimating pixel weights that respective pixels of the image signalincluding distortion have when producing an undistorted version of theimage signal; (ii) scaling pixel intensity values by respective pixelweights to produce a weighted pixel value for the respective pixels; (b)provide the weighted pixel values to an exposure control pixel countingprocess to produce one or more exposure feedback control signals; and(c) transmit the one or more exposure feedback control signals to atleast one of the light emitting element, the optical assembly, and theimage sensor assembly.
 20. The circuitry of claim 19 in which theexposure control circuitry is further configured to receive a new angleat which the imaging device will operate and, in response, activate asecond distortion model for performing exposure control at the new angleof operation, and receive a stream of image frames produced at the newangle.